This invention relates to sensor technology. In particular, the invention relates to a tool or device for measuring the geometric location and configurations of objects in space. The invention is suited to robotic applications and to monitoring or measuring human geometry and motion. This invention represents modifications and improvements in the inventions described in U.S. Pat. No. 6,127,672, issued Oct. 3, 2000 to Lee Danisch, and PCT Appln. No. PCT/CA98/00213, published as WO98/41815, which are both incorporated by reference.
A preferred application, amongst others, is in the field of animation effected by motion capture of movements by the human body.
Various technologies have been applied to measure the location, orientation and surface shapes of objects in space.
In the field of robotics it is known to determine the location of a flexible member formed by a series of rigid, linked elements in space by measuring the angular degree of rotation existing at the various joints joining such linked elements.
Rotations in a flexible member include bending that is transverse to the longitudinal extent of the member; and twisting that occurs about an axis which is coincident with the longitudinal extent of the member. Both types of movement or distortions qualify as xe2x80x9cflexuresxe2x80x9d.
Twist is usually negligible in sensor structures based on cylinders, rods, and other solids with significant cross-sectional dimensions. However, it can be very advantageous to measure the presence of twist in elongate, flexible members that are rope-like in their flexibility. Such members would be very convenient for incorporation in garments.
Thus one object of the present invention is to provide an improved flexible member or reference platform equipped with distributed sensors wherein changes in the shape of the member are sensed by the sensors in such a way that the complete shape of the member can be found by calculations from the outputs of the sensors.
Another object of this invention is to provide an instrumented flexible member that is sufficiently compliant to substantially conform to the surface of a curved object and act as a sensor to provide electronically processable data as to the shape of that surface.
A variety of sensors exists for measuring the state of flexurexe2x80x94bend and twistxe2x80x94in an object; these include optical fibers and conductive metal fibers, i.e. wires. A convenient class of sensors particularly suited to this objective relies on fiber optics.
U.S. Pat. No. 5,321,257 to Lee Danisch, describes modified optical fibers that are provided with a light absorbent region on a portion of the outer fiber surface, especially on one selected side of the outer fiber surface, such region providing a bend sensor whereby the curvature at such modified region may be remotely detected by the change in the overall light transmission capacity of the fiber. This prior patent depicts, in FIG. 12, the deployment of clusters of modified fibers capable of detecting a bend at a particular location in three dimensional space. The associated fiber ends are all connectable at one end to a multi-fiber light source, light sensing and signal processing unit. These modified fibers, or so-called xe2x80x9cbend enhanced fibersxe2x80x9d, are referenced in the aforesaid ""672 patent.
The aforesaid U.S. Pat. No. 5,531,257 also discloses three optic fiber sensors mounted in parallel, with the sensors being sensitive to bending in separate directions, and which are used for resolving bends in multiple DOFs in a flexing structure. However, this prior patent does not suggest any method of dealing with twist, which would cause ambiguity or be undetectable in the readings of the patented sensor method of this ""257 patent. Accordingly, this patent does not deal with the problem of determining the complete position and orientation of a longitudinally extended structure based only on measurement of flexure.
Another problem with this simple structure is that when three straight fibers are bent, some will be extended and some compressed, due to difference in radius of curvature, which leads to significant errors in measurement.
A further paper on this subject by the inventor herein entitled xe2x80x9cLaminated Beam Loopsxe2x80x9d has been published in SPIE Vol. 2839, pp. 311-322, 1996. The contents of this paper, the above referenced United States patents and the published PCT application PCT/CA94/00314 are all incorporated by reference herein.
Optical fiber sensors can measure bend and, in accordance with the invention described in the aforesaid ""672 patent, twist, based on the disposition of the fiber and the location of the treated, light absorbing region of the fiber surface along the fiber. The sensitive region of the fiber can be contained within a running length of on the order of three millimeters to many centimeters for example 30 cm, depending on desired sensitivity and the dimensions of the fibers. This provides a corresponding span for the sampling of the average state of curvature of the sensing region of the optic fiber. Fiber optic technology is convenient for use in sensors because it is robust, benign and inexpensive. The aforesaid ""672 patent describes various forms of measuring tools which incorporate bend and twist sensors, of which the preferred forms are fiber optic based sensor systems that can provide remote information on the locations of objects in space, the shape of surfaces and changes in the shape of surfaces. However, the ""672 patent is not limited to fiber optic based sensors. Similarly, while the present invention will also be described principally in relation to fiber optics sensors, it is not limited to systems using such sensors.
The invention of the U.S. ""672 patent, according to one aspect, is a shape and position measuring tool which comprises the following features:
1. a longitudinally extending, flexible substrate having a compliant reference surface and being capable of bending in at least two degrees of freedom so as to be configurable in three dimensional space;
2. a plurality of spaced bend sensor means and a plurality of spaced twist sensor means each of said plurality of sensor means being respectively coupled to and positioned at specific discrete locations, on and at known respective bend sensor and twist sensor spacing intervals along the longitudinal extent of the substrate, to provide flexure signals indicating the respective local state of bend and twist present in the substrate at the respective locations where the respective bend sensor means and twist sensor means are coupled to the substrate;
3. sensor data processing means coupled to the bend sensor means and the twist sensor means for receiving signals therefrom and for presenting data on the geometric configuration of the reference surface of the substrate in three-dimensional space,
wherein the sensor data processing means operates by determining the geometric configuration of the substrate from the bend and twist signals derived from the flexure signals provided by the bend sensor means and twist sensor means at their specific locations and from the spacing intervals between such sensor means.
As indicated, the shape measuring tool of the ""672 patent included a flexible substrate which carried the bend and twist sensors at specific locations on the substrate. The substrates included ribbons (FIGS. 5, 6 and 8), and ropes (FIG. 7). Substrates of sheet form were also contemplated, including a planar array with substantial width as well as length, for example a xe2x80x9ckeyboardxe2x80x9d type of device.
The present invention is based on similar principles, but with the omission or minimization of the presence of a substrate for the support and positioning of flexure sensors.
The invention in its general form will first be described, and then its implementation in terms of specific embodiments will be detailed with reference to the drawings following hereafter. These embodiments are intended to demonstrate the principle of the invention, and the manner of its implementation. The invention in its broadest and more specific forms will then be further described, and defined, in each of the individual claims which conclude this Specification.
It has now been found possible and in some cases advantageous to construct flexible members similar to those described in the ""672 patent but wherein there is no substrate, or only a minimal substrate, additional to the fibers which provide the sensors.
Accordingly, the present invention provides in one aspect a measuring device for providing data corresponding to a geometric configuration in space, the device in one aspect being in the form of a flexible, compliant, measurement member capable of bending in at least one degree of freedom, the member extending along a medial axis or plane and having spaced flexure sensors distributed at sensing locales having known locations on the member and separated by known sensor spacing intervals to provide flexure signals indicating the local state of flexure present at said sensing locales.
According to one embodiment the measurement member comprises a multiplicity of formed fibers, as defined below, said formed fibers including sensing fibers having sensing portions which provide said flexure sensors, the sensing portions of different fibers being located at sensing locales at differing distances along said member so as to be located at said sensor spacing intervals, said formed fibers being in mutually supporting relationship, the overall form of the measurement member being substantially maintained by the form of the constituent fibers themselves or by their continuous or repeated contact with each other to enhance the strength or stability of the member. The fibers may have continuous or repeated connections with each other, such as by being interwoven or entwined to provide dimensional stability. Alternately, the fibers may be sufficiently rigid to be self-supporting without inter-fiber contact. The xe2x80x9cformxe2x80x9d of the device refers to the organization and placement of the fibers with their sensing portions, allowing that measuring device is, overall, flexible and capable of conforming to various shapes in space.
Such fibers are preferably obliquely deployed or formed in the sense that their sensing portions are obliquely oriented with respect to the extent of the member itself.
The term xe2x80x9cfiberxe2x80x9d in this context means an element of a member with primarily axial extent, its path describable by a space curve, which is a well-defined mathematical entity with mathematically defined bend and twist. The term xe2x80x9cfiberxe2x80x9d includes an element of polygonal cross-section.
The term xe2x80x9cobliquely formedxe2x80x9d in relation to fibers, or xe2x80x9cobliquely orientedxe2x80x9d in relation to sensing portions, means being non-straight or non-aligned with the medial axis or plane of the member when the medial axis or plane of the member is straight or flat, i.e. fibers have a two-dimensional or three-dimensional form when the overall member is straight or flat. These formed fibers are normally deployed in a cyclical or repeating pattern. Formed fibers may either be fibers such as optical fibers which have been heat treated while being deformed so as to have a permanent shape, or they may be non-straight merely by virtue, for example, of being twisted into the form of a rope or knitted or woven into a textile.
Formed fibers have the following useful properties for this invention:
a) Formed fibers have an ability to support each other by repeated contact or repeated connections amongst the fibers and along the member constituted by the fibers. This ability to support each other despite flexure of the member in one or more degrees of freedom preferably persists over a large angular range of both bend and twist. The mutual support between the fibers, which gives the member an integral structure, distinguishes the member of this invention from:
1) a mere bundle of loose fibers, and
2) multiple straight fibers held in a non-planar bundle by adhesives
b) The formed fibers can readily form cyclical structures, i.e. structures with repeating patterns, for example ropes, ribbons formed by adjacent wavy fibers, and woven or knitted textile type structures. Such structures may have cyclically repeating curves that take on cyclically repeating pairs of opposed deformations locally within each curve during curvature of the measuring device, without substantial changes in the net extension or compression of the fibers along the full extent of the device whereby said curvature can occur without overall slippage of fibers along their full length. Preferably, the fibers are optical fibers with loss zones having a circumferential orientation and axial placement along each fiber, said orientation and axial placement producing a desired circumferential orientation and axial placement of light loss with respect to the device resulting from the formed curves of the fibers, such that the loss geometry produces modulation of the light indicative of the position and orientation of the device.
c) Structures made of the formed fibers have an ability to flex over a large range of bend and twist without significantly slipping past each other either locally or over their complete extent, and without a substantial net extension or compression of the fibers axially along the member. When used in a rope, out-facing waves of one fiber are extended under tension, while in-facing waves of the same fiber compress, with the net extension being zero. Local slip is minimal and can be designed to be within the elastic limits of an adhesive, if any is used.
As an example of a two-dimensional (when flat) form of the invention, a multiplicity of formed fibers in the configuration of a wavy ribbon (wavy within its plane) may be used as a measurement instrument. Such ribbon may be affixed to a moving body and will follow the movements without buckling, because the formed waves are able to bend slightly within the plane of the ribbon throughout their lengths, thereby absorbing length differences inherent in measurements outside the neutral axis of a body. Similar attachment of an unwaved ribbon will result in buckling of the ribbon during axial compression. The waves also permit sensing of twist, which is impossible if the fibers are purely axial in lay.
Another aspect of the invention when the member is of an undulating or cyclical form is that flexure results in changes in the proximity (edge-to-edge) and register (axial loci change relative location axially along the fibers) between adjacent cycles of the formed shape, for example a helical shape. This occurs in cyclical fashion along the overall member, without an overall change in length of the fibers.
d) Formed fibers according to the invention provide the ability to sense twist, since the fibers have sensing portions which are obliquely oriented to the axial direction or plane of the member. A suitable structure may be made particularly sensitive to twist by having a wind-up/wind-down portion of the member that has extra twist present over a limited span that includes sensing portions. Such a structure may be tailored to be minimally responsive to bend, by adjustment of the extra-twist portion. A twist-sensitive portion of fiber gives a maximum response if oriented 45 degrees to the medial axis or plane of the member.
e) the invention provides a means of treating the fibers in their formation so that the imposed form imparts to treated zones desired three dimensional angular orientations within the member and bipolar response to curvature of the member. This can be made to occur even if the same zones had only axial orientation and unipolar response to curvature, when straight and flat.
In some circumstances, all the formed fibers extend substantially the full length of the member. In fact, the member may be made up entirely or largely of the formed fibers, so that the stiffness of the member is not substantially greater than the combined stiffness of the fibers. Similarly, the formed fibers may provide the major part of the tensile strength of the member. In particular cases, formed optical fibers which are sensing fibers, i.e. have sensing portions, may constitute a large portion, or virtually the whole of, the member.
It will be shown in this disclosure that cyclical structures enable an advantageous interplay between the cyclical form and the functions required of sensors on the fibers. These advantages hold even for structures that do not support each other as in a) above, but rather are relatively stiff so that the structure is a grouping of cyclical fibers held in interrelation by end supports. An example is three helixes like helical springs with a common central axis, held at the ends, for example, by disc-shaped plates, but that do not touch each other. Such a structure can bend and twist, has formed fibers, and provides an improved platform on which to place sensors. These structures are distinguished over the prior art in part by the fact that at least most of the formed fibers extend substantially the full length of the member so as to contribute materially to the strength of the member, and in one variant, no structure other than the fibers is necessary to maintain structural integrity.
In some cases, formed fibers, including sensing fibers, may make up most or all of an elongated member, but not all the formed fibers need extend the full length; this results in a tapered member in which each fiber extends just as far as is necessary along the member, to the point where it is required to sense bending or twist.
Generally, the member is elongated and the sensors are spaced along the member at said locations. In order to measure twist as well as bending, portions of at least some of the fibers which constitute the sensors are obliquely orientated to the longitudinal dimension of the member, preferably at 45 degrees, to the member medial axis for an elongate member. However, the member may also be in the form of a flexible sheet, with the sensors spaced all over the sheet. In such case, the sensing portions of at least some of the fibers are obliquely oriented either with respect to each other or with respect to the plane of the member.
For the elongated member, the invention works by sampling curvature at multiple, spaced intervals along the elongated member. The invention relies upon inter-referencing the position of flexure sensors located at known intervals along the member with the location of adjacent sensors so that the location of all sensors with respect to each other is known.
Bend can be measured about either one or two axes that are orthogonal to the longitudinal dimension of the member, depending on the nature of the member. Thus a measurement member of rope-like form would require that bending be sensed about two such axes, either directly or indirectly.
By providing a member which is deformable only in restricted degrees of freedom, the number of sensors required can be reduced. In one preferred configuration, the member is in the form of a ribbon formed of a series of fibers, for example optical fibers, connected together side-by-side. In such case bend sensors may be the only sensors required for measuring flexure of the ribbon in its permitted bending mode. This reduces the number of bend sensors needed per unit of length.
A ribbon is an article which is substantially limited to bending along its length about axes which are transverse to the longitudinal dimension of the ribbon and within the plane of the ribbon, while the ribbon remains free to twist about its longitudinal dimension. Thus a single bend sensor will suffice to measure bend at a location along a ribbon. To complete the definition of the geometric configuration of the ribbon-like member, twist must also be measured by twist sensors located at known intervals along the longitudinal extent of the length of the ribbon. Such bend and twist sensors may be interspersed with each other or be co-located along the ribbon. The ratio between such types of sensors may depart from a strict ratio of 1:1. Furthermore, in many cases each individual sensor is responsive to two or more degrees of freedom, such as two degrees of freedom of bend and one of twist, and the degrees of freedom are separately determined by mathematical operations involving all of the sensors in a similar region.
When the member is of a ribbon-like configuration employing both bend and twist sensors, freedom of movement and tracking of the geometric configuration of the ribbon in three dimensional space is nevertheless available. This is because the ability of a ribbon to twist allows portions of the ribbon to be re-oriented in any direction in space.
An elongated configuration for the invention can also be implemented by applying an instrumented planar fibrous tape of fibers assembled in the ribbon-type format helically to the outside, or inside, of a cylindrical flexure of resilient material, for example a hose. When a hose-like carrier is employed, sensor communications may pass through the core. While a carrier may contribute to the strength or stability of the member, the oblique orientation of the sensing portions of the fibers allow for bend and twist conditions to be monitored along the length of the member. The fibrous tape in such case contributes materially to the overall strength or stability of the composite member, preferably the greater portion. In the case of a helically coiled fibrous tape or ribbon, according to the invention, used in conjunction with a tubular carrier, counter-rotating helical components may be combined as in the structure of a) braided rope, or b) a layer wound clockwise, surmounted by a layer wound counterclockwise. This further enhances the mechanical stability of the structure, such as making it resistant to twisting.
The invention is an improvement over previous forms in that the fibers are either a) on the neutral axis of the member as in the case of a ribbon with side-by-side fibers or b) bend and twist without net elongation as in the case of rope and tube forms where outward-facing curves extend and inward-facing curves compress during curvature of the member.
Where the bend and twist sensors are sensing portions of optical fibers that have been rendered sensitive to their state of curvature as described above, the optical fibers which form the member, or a main part of the member, will usually have only one such sensing portion on each fiber, the positions of which are strategically arranged to sense bending at appropriate places on the member. However, the light sensing portions may be made specific to certain light wavelengths, and may be connected to light source and signal processing units which provide the different wavelengths and which distinguish between them, in which case each fiber can have several light sensing portions at different positions along its length and can produce information relating to bend at the different positions.
In the case of a ribbon that is generally constrained to exhibit bending about axes extending transversely to the ribbon""s length, the sensors may be portions of optical fibers that are aligned parallel to the plane of the ribbon and in the direction of its longitudinal length at the locations where bending information is required. The sensing portions of an optical fiber may be generally aligned to lie across the axis about which bending is to occur, e.g. the axes extending transversely to the length of a ribbon.
For convenience of signal processing in a ribbon member, both twist and bend at a single location can be measured using two bend sensors having their bend sensing portions oriented at angles with respect to each other. These bend sensors may or may not be looped optical fibers, as described in PCT Application PCT/CA94/00314 (published as WO 94/29671), and in the aforesaid ""672 patent. The directions of the sensing portions of the pair of fibers are preferably oriented at substantially the same angle off the longitudinal median line of the ribbon member and preferably at 45 degrees to the longitudinal dimension of the member for maximal effect. This permits two fibers to be used to measure both bend and twist at a single location by processing their outputs to extract their sum and difference signals as a measure of twist or bend. The referenced angular orientations simplify signal processing. With computational adjustments other angles would still be able to provide both twist and bend values from a splayed pair of sensors. Since the sensors will normally be operated in their linear ranges, the computations normally involve sums and differences of linear equations which are very amenable to high speed automatic computation.
By assembling distributed sets of sensors, flexure sensing regions may be formed not only linearly, as along a supporting rope or ribbon-like member, but also over an area of a flexible sheet-like member. The sensor portions can be in groups and can consist of bend and twist sensors, or dual-direction bend sensors, which are able to completely describe the shape of the sheet. Using data on the state of curvature at each sensing region, and knowing the separation between sensors, the signal detection system can construct a depiction of the shape of the sheet. With the sheet placed in contact with a geometric surface of unknown form, the shape of such surface can be measured, at least where the sheet and surface are in contact.
In all forms, the sensors for bend need not be co-located with the sensors for twist, and/or bend sensors need not be co-located with their differently oriented bend sensing mates. It is sufficient for them to be distributed along the member at known intervals that allow the configuration of the member to be determined.
Although most references herein have been made to inextensible flexures, extensibility can be allowed to exist in the member. Thus, a possible form of the invention could be a stretchable member wherein not only bend and possibly torsion are measured but also extension. The degree of extension must be detected to ensure that the spacings between the flexure sensors will be known. Extension sensors could include portions of conductive elastomers sensitive to extension. For convenience and to improve compliance, extension could be limited to a small increase in length beyond which the flexure becomes functionally inextensible. Alternatively, in a xe2x80x98doubly formed embodimentxe2x80x99, a sensor body with an elongate form (e.g. a tape, rope, rod, etc.) containing formed fibers with loss zones to sense its bend and twist is further formed (e.g. by heat treatment or adhesives) to follow a sinuated path, e.g. a helical path. The extension and compression of the resulting doubly formed member will be measured along with the other details of its shape as a consequence of measuring the aforesaid bend and twist and thereby determining position and orientation of all portions of the elongate sensory member.
In sensing members of this invention, formed largely or entirely of formed fibers, one of the most useful forms for bend and twist sensing is the helix, which is a cyclical 3 dimensional structure defined by constant bend and twist along its length when undeformed (unbent). Other useful forms for sensing members of the invention include loops and waves which are primarily 2 dimensional structures. Cyclical flexures provide many opportunities for exploiting shape sensing.
Many constructions are based on the helical form. A good example is rope, which comprises fibers wound into strands, which are in turn wound into the rope. The elastic tendency of the fibers to unwind from the strands is counteracted by winding the strands into rope in a direction opposite to that used to wind fibers into strands. Then any unwinding of the fibers tends to tighten the strands within the rope. Forces and moments within the system reach equilibrium to maintain the structure of the member. Another example of use of the helical form is the helical covering for wires and cables, formed from a bent and twisted band of metal or plastic.
Ropes, and other helical forms, are examples of cyclical members or flexures made up of formed fibers which provide repeating zones with similar orientations. This allows bend or twist sensing to be distributed along an extent of a single sensor fiber (or pair, or triplet, or larger subset of fibers at a known location) of a larger array of such subsets. The single sensor or grouping can take on the repeating characteristic, such as an orientation of 45 degrees to an axis, without requiring that all the fibers forming the entire member have that orientation.
Most cyclical structures provide flexibility while permitting significant lateral deflections. This is true of sinuous ribbons, ropes, and textiles (2D elongate, 3D elongate, and 2D or 3D elongate or planar structures respectively). A sinuous or wavy ribbon, i.e. wavy within its plane, which may be made by glueing sinuous fibers together, will be able to extend and compress during flexing of the body because the individual waves can extend and compress by bending within their planes. A helical rope can be twisted into a larger helix and a sinuous ribbon can also be twisted into a helix. The lateral deflections provided by cyclical structures are useful when the sensor must surround another body, such as a flexible endoscope whose shape is to be sensed, or carry elements such as wires or return fibers within its structure, or when the sensor has a multiplicity of fibers or wires that take up space. The combination of flexibility and lateral deflectability may be exploited to a high degree in sinuated, looped, and helical structures.
Ropes and textiles were invented millenia ago to conform to arbitrary surfaces, and employ the same sinuations, loops, helixes, braids, and woven or knitted elements that will be demonstrated herein for sensor structures which use formed fibers at least some of which are sensing fibers. Significant improvement over straight fibers attached to flexible substrates is effected by the use of cyclically repeating, sinuous fibers that form a sensing member through interrelationship without a separate substrate. Examples include ropes and textiles with sensing capabilities along the fibers.
Unless the sensing fibers (optical, wire, etc.) are at the neutral axis, some slippage between the fibers and any substrate used, in addition to the fibers, must be permitted to occur or bend and twist sensing will be compromised. This is because a length change must be accommodated for any in-extensible/incompressible flexure lying against or upon another similar flexure, when both are bent: both flexures can accommodate the same radius of curvature only if allowed to slip one upon the other. If slip is not allowed to occur, either one of the fibers will buckle in compression at the point of weakest attachment, or bend in a plane will be converted to twist or out-of-plane bending of the fiber. These problems can largely be avoided by forming the member from the fibers themselves, without any additional substrate.
The foregoing summarizes the principal features of the invention and some of its optional aspects. The invention may be further understood by the description of the preferred embodiments, in conjunction with the drawings, which now follow.